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Clickers, electronic response systems, are widely popular in college classrooms and 
proponents have argued clickers can increase student engagement, active learning, 
and, perhaps most importantly, student comprehension. Determining whether the 
effectiveness of clickers justified their purchase seemed warranted. A campus-wide 
project was developed to address this question. Five university instructors adopted 
clickers in one of their introductory courses (psychology, speech, accounting, and 
education) while teaching a second, control course. Contrary to expectation, 
attrition was higher and grades were lower in the clicker courses, although these 
differences were not statistically significant. 

Over the past 20 years, the technology used on university campuses has 
expanded from overhead projectors and videotapes to comprehensive multi-media 
presentations involving laptops, LCD projectors, online testing, and personal 
response systems or clickers (MacGeorge, et al ., 2008; Stowell & Nelson, 2007). 
Researchers report that instructors may use the novel technological capabilities of 
clickers to enhance questioning and feedback (Trees & Jackson, 2007); to motivate 
and monitor the participation of each of their students (Stowell & Nelson, 2007); to 
foster discussions of important concepts (Brickman, 2006); and to energize and 
activate students' thinking (Collins, Moore, & Shaw-Kokot, 2007). Bransford and 
colleagues cite clicker technology and the related pedagogy as one of the most 
promising methods for transforming classrooms to be more learner-, knowledge-, 
assessment-, and community-centered (Bransford, Brophy, & Williams, 2000). 

In addition, students appear to favor electronic response systems over 
traditional lecture formats. Judson and Sawada (2002), in a review of the literature 
on clicker use, noted that students find clickers to be helpful in comprehension, and 
cited some (although limited) evidence for benefits in academic achievement. In a 
study of clicker use in biology classes, Brewer (2004) found that the use of clickers 
allowed instructors to receive feedback that helped them appropriately set the pace 
of the course. Clickers can also enhance reflection and understanding when used 
with small group discussion (Brewer, 2004; Brickman, 2006). Likewise, Draper and 
Brown (2004), in a multi-disciplinary study of clicker use, note several advantages 
of clickers (lectures are more fun, anonymity allows students to answer without the 
risk of embarrassment, students can check their understanding of the material). 

However, students and instructors both have cited some possible 
disadvantages of clicker use (clickers can distract from learning, focus seems to be 
on technology rather than the material, questions are not very helpful). Carnaghan 
and Webb (2007), for example, found that student engagement declined when 
clickers were introduced into their courses. Although students reported enjoying the 
use of the clickers, this satisfaction did not translate into increased satisfaction with 
the course. Additionally, strong evidence for 
increases in test scores and/or course grades 
associated with the use of clickers is minimal 
as yet (Draper & Brown, 2004; Judson & 

Sawada, 2002). The research to date seems 
to suggest it is how the instructor makes use 
of the clickers, rather than the simple 
adoption of clickers themselves, that 
determines their pedagogical effectiveness. 

Reviewing the literature on clickers suggested that the most positive 
results of clickers were posited to occur in relatively large classes where one-on-one 
interaction between students and professors may not be feasible. Draper and Brown 


The research to date seems 
to suggest it is how the 
instructor makes use of the 
clickers, rather than the 
simple adoption of clickers 
themselves, that determines 
their pedagogical 
effectiveness. 


InSight: A Journal of Scholarly Teaching 


31 



(2004), for example, assessed the student response to clickers in fifteen classes 
ranging in size from 18 students to close to 500 students. They argued that clickers 
allowed increased interactivity during lectures and that even students in the 
smallest classes appreciated the increased anonymity afforded by the use of 
clickers. More recently, Morling, McAuliffe, Cohen and DiLorenzo (2008) assessed 
the efficacy of clickers in four large sections of introductory psychology, with 
approximately 350 students per section. These researchers reported that clickers 
led to a small, positive effect on exam scores. However, students in the clicker 
classes in this study did not report feeling any more engaged during class than did 
students in the non-clicker classes. 

As these results have become more well-known, the faculty on our campus 
began to consider using clickers in their classes. Our teaching and learning center 
began to receive requests to provide 
workshops on the use of clickers and, 
individually, several faculty members began 
contacting publishers about the possibility of 
adopting clickers. It appeared that we were on 
the 'fast track' to adopting this new 
technology. 

The primary mission of our university 
is teaching and, as a result, we have 
maintained an average class size of 
approximately 25 students. Prior to asking students to purchase clickers for their 
classes, we wanted to determine if this technology would be pedagogically effective, 
given our small class size. Although familiar with the rationale behind the 
scholarship of teaching and learning, many faculty on our campus were less 
comfortable in attempting to assess whether clickers increased student learning. 
Therefore, with the help of our teaching and learning center, five introductory 
courses were selected to introduce and evaluate the use of clickers. The goal of this 
research was to identify whether the addition of clickers would improve retention 
rates, grades, and student satisfaction. Five instructors each taught two sections of 
their respective introductory course, one section using a standard lecture format, 
the other incorporating personal response systems. The course objectives, 
materials, and grading practices were identical for each section the instructor 
taught. It was expected that the sections using clickers would have lower attrition 
levels, higher grades, and greater satisfaction with technology use. 

Method 
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Participants 

Five full-time residential faculty members (one full professor, two associate 
professors, one assistant professor, and one senior lecturer) were the instructors for 
the selected courses. Students in ten sections of introductory courses (four sections 
of psychology, two sections of accounting, two sections of education, and two 
sections of speech) agreed to participate in this study, which had been approved by 
the university's Institutional Review Board (IRB). No course credit or other benefits 
were provided for participation. Enrollment in both the control and clicker classes 
varied from approximately 20 students per course to approximately 50 students per 
course, with the average enrollment being 30 students for the control classes and 
29.2 students for the clicker classes. Overall, 146 students participated in the 
clicker classes and 150 students in the control classes. Despite this across-instructor 
disparity, each instructor's clicker and control conditions were quite similar (see 
Table 1). Mean age and gender breakdown were fairly consistent and representative 
of the campus. Overall, 70% of students were in their first year of college, 18% 
were sophomores, and 2% were juniors. No differences in class rank were found 
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between control and clicker classes. In addition, no differences in high school rank 
or SAT scores were found between the classes. 

Table 1: Demographics of Students in Control and Clicker Classes 



Mean Age 

Percent Females 

Percent Males 

Clicker 

21.7 

66% 

34% 

Control 

21.3 

73% 

27% 


Materials 

Instructors used their standard lecture format for both the clicker and the 
control classes. All assignments, exams, readings, and material covered were as 
similar as possible in both the clicker and the control condition for each instructor. 
For the clicker classes, students were asked to purchase from the university 
bookstore the university supported clicker, the elnstruction CPS radio frequency 
electronic response remote. This remote allows students to enter either numbers or 
letters for multiple-choice, true-false, or instructor-created questions. Each 
instructor created his or her own questions for use with the clickers. Instructors 
attended a week-long training session sponsored by the campus teaching and 
learning center, during which they were taught how to use the clickers and 
practiced creating questions that would focus on building conceptual understanding 
and opportunities for discussion, rather than simply reflecting memorized items. In 
addition, instructors were informed of the design methodology being used to 
evaluate clickers in the present study. 

At the end of the semester, all students completed a course survey 
measuring study habits and attitudes toward the class. Questions included items 
such as how much time students studied outside of class, how much time students 
spent preparing for the class, whether students actively participated in class, and 
whether the instructor's use of new technology promoted learning in the course or 
created anxiety for the student. Each of the five instructors completed a similar 
questionnaire to assess how much time they spent preparing for the class and their 
attitudes. 

Design and Procedure 

All five faculty members signed informed consent statements allowing 
research data from their courses to be used. For each of the five instructors, one 
section of their introductory course was assigned to the clicker condition, and the 
other was assigned to the control condition. All other elements of the course were 
held constant including testing, assignments, readings, and material covered. 
Clickers were used during class sessions. Questions varied from opinion questions to 
reading checks to questions designed to create discussion or check students' 
conceptual understanding of topics being covered. All questions asked were 
formative; that is, no question was used as a summative, graded part of the class. 
Students responded using the clickers and then the material was reviewed as 
needed. At the end of the semester, all students were asked to complete a course 
survey designed to measure study habits and attitudes toward the class. All 
students were informed of the nature of the study and signed informed consent 
forms. 


Results 

The primary dependent variables in this study were student attrition and 
final class grades. Additionally, faculty and student comments about the class, 
collected with the end-of-course survey for the students, were also assessed. These 
comments were submitted on standardized end-of-course evaluation forms. 
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In the clicker classes, 24.66% of students failed to complete the course, 
compared to 15.33% of students who failed to complete the control classes (see 
Table 2). Chi square analysis revealed that this difference was not significant. Grade 
distributions also varied between the clicker and control classes, although Chi 
square analysis revealed that these differences were not significant. As can be seen 
in Table 3, students in the clicker classes earned more Cs and Fs than students in 
the control classes. Likewise, students in the control classes earned more As and Bs. 
The percentage of students earning Ds was identical in both the control and clicker 
classes. 


Table 2: Attrition in Control and Clicker Classes 



Enrollment 

Completion 

Attrition 

Rate 

Clicker 

146 

110 

24.66% 

Control 

150 

127 

15.33% 


Table 3: Percentage of Final Grades in Control and Clicker Classes 



A 

B 

C 

D 

F 

Clicker 

30% 

26% 

25% 

8% 

11% 

Control 

35% 

28% 

21% 

8% 

8% 


Questions from the end-of-course evaluations did not reveal any apparent 
differences between the clicker and control classes. Many students also provided 
written comments about the clicker class on the end-of-the course evaluations with 
70% of those students responding reporting that they enjoyed using the clickers. In 
addition, 42% of students reported that they enjoyed the anonymity of the clickers. 
The remaining students were less positive, reporting that they did not like having to 
pay for the clickers (65%) or that the clickers seemed to interfere with discussion in 
the classroom (74%). 

Faculty comments revealed both positive and negative aspects of the use 
of clickers. On the positive side, all faculty reported that they enjoyed using the 
clickers and believed that students enjoyed the clickers as well. Two negative 
comments were reported. First, the clickers required more time both in preparing 
for the class and in conducting the class. When teaching clicker classes, faculty 
reported spending an average of 2.6 hours more per week preparing for class than 
when they taught the control classes. Within the class, three of the five faculty 
reported they had difficulty covering the same material as in the control classes due 
to the increased time required for students to use their clickers. A second negative 
comment, from three of the five instructors, concerned what they labeled the 
'camaraderie of dissent.' According to these three instructors, clickers allowed 
students to see how many of them did not understand the material, leading to their 
rationalization that the material the professor was covering was simply too hard. 

Discussion 


In the present study, no significant differences were found in attrition or 
grades between the clicker and control classes. Attrition rates were higher in the 
clicker classes, but this difference was not 
significant. The reason for this is unknown, 
because students who withdrew obviously did 
not complete the end-of-course evaluation. 

Perhaps these students left because they 
disliked having to purchase or use the clickers. 

Neither were there significant differences in 

grade distributions across the two types of classes. This is of concern, as one 
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argument for asking students to purchase clickers is that their use is thought to 
improve student learning. This did not occur in the present study. 

Several factors in the present study may have led to these results and may 
provide the basis for recommendations on the best use of clicker technology. First, 
in many universities, clickers are being used in large introductory sections of 
classes. In these classes, faculty interaction with students may be limited due to the 
sheer number of students and the necessity for the professor to cover a large 
amount of material. In such classes, students may be able to skip class or attend 
but never say a word, so the use of clickers guarantee at least some level of 
participation. In smaller classes, such as the ones in the present study, students are 
more likely to already have significant interaction with the professors. In fact, 
several students in the present study reported that they disliked the clickers 
because they interfered with the type of interaction they were accustomed to with 
their professors. These results are consistent with those of Carnaghan and Webb 
(2007), who also found a decline in student engagement with the introduction of 
clickers. As a result, one recommendation might be to only use clickers in large 
classes where more personal means of interaction might be problematic. 

A second issue concerns the familiarity of the faculty member with the use 
of clickers and the manner in which the faculty member incorporates clickers into 
the classroom. The first time a faculty member uses a new form of technology or 
introduces any new component into a course, there may be awkwardness leading to 
increased problems and less chance of success (Draper & Brown, 2004). In the 
present study, this was the first time the instructors had utilized clickers in the 
classroom. It is certainly possible that with continued use, clicker classes would 
show the expected improvements in learning suggested by the literature. This may 
not be a strong explanation in the present study, however, because all five 
instructors went through a clicker training process aimed to both reduce any 
awkwardness in use, and ensure that questions created for use with the clickers 
would be best suited for student engagement and learning. In fact, the fifth 
instructor continued to use the clickers for two additional semesters and continued 
to collect data. The results in her subsequent classes matched the results during the 
original semester. Whether faculty using clickers in other locations receive such 
training prior to using this technology is questionable. To ensure success with 
clickers, it seems likely that professors need specific training in the use of the 
technology and in writing appropriate questions for use with the clickers. Questions 
of most use might be those designed to enhance discussion and those that probe for 
more conceptual or applied understanding on the part of students. For example, 
research has noted the probable benefits of clickers when used to facilitate students' 
"interactive engagement" with the material, the instructor, and each other (Brewer, 
2004; Draper & Brown, 2004; Judson & Sawada, 2002). 

Finally, in the present study, clickers were only used for formative 
purposes. No grades were attached to 

students' responses to clicker questions. It Using a cross~campus 

seems reasonable to expect differences to approach to designing and 

appear when faculty use formative vs. conducting research on 

summative questions. It is possible that if the clicker effectiveness allowed 

instructors in the present study had combined for the inclusion of faculty 
formative and summative clicker questions, who might never have 
the students would have valued this aspect of considered conducting such 
the class more and students in the clicker research on their own. 
sections would have performed better in the 
class. This question would be an interesting one for future research. 

From a broader perspective, this research led our campus to several 
decisions. While not prevented from utilizing clickers in their classroom if they so 
chose, faculty were encouraged to consider the results of the present study and to 
attend the training provided by the teaching and learning center. Four of the five 
instructors in the study decided not to use clickers in the future. As mentioned 
earlier, the fifth instructor continued with the clickers for two additional semesters, 
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but due to continuing poor results, has now discontinued their use. Using a cross- 
campus approach to designing and conducting research on clicker effectiveness 
allowed for the inclusion of faculty who might never have considered conducting 
such research on their own. In addition, this model has been adopted by our 
campus for assessing the effectiveness of other forms of pedagogical technology, 
prior to their widespread introduction across the campus. 
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